<!--begin.rcode random_load, results='hide', echo=FALSE, message=FALSE
library(caret)
library(pROC)
library(nnet)

library(parallel)
library(doMC)
registerDoMC(cores=detectCores()-1)
options(width = 100, digits = 4)
theme_set(theme_bw())

opts_chunk$set(comment=NA, digits = 4,warning=FALSE,message=FALSE)


session <- paste(format(Sys.time(), "%a %b %d %Y"),
                 "using caret version",
                 packageDescription("caret")$Version,
                 "and",
                 R.Version()$version.string)
    end.rcode-->

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
  <!--
  Design by Free CSS Templates
http://www.freecsstemplates.org
Released for free under a Creative Commons Attribution 2.5 License

Name       : Emerald 
Description: A two-column, fixed-width design with dark color scheme.
Version    : 1.0
Released   : 20120902

-->
  <html xmlns="http://www.w3.org/1999/xhtml">
  <head>
  <meta name="keywords" content="" />
  <meta name="description" content="" />
  <meta http-equiv="content-type" content="text/html; charset=utf-8" />
  <title>Random Hyperparameter Search</title>
  <link href='http://fonts.googleapis.com/css?family=Abel' rel='stylesheet' type='text/css'>
  <link href="style.css" rel="stylesheet" type="text/css" media="screen" />
  </head>
  <body>
  <div id="wrapper">
  <div id="header-wrapper" class="container">
  <div id="header" class="container">
  <div id="logo">
  <h1><a href="#">Random Hyperparameter Search</a></h1>
</div>

  </div>
  <div><img src="images/img03.png" width="1000" height="40" alt="" /></div>
  </div>
  <!-- end #header -->
<div id="page">
  <div id="content">

      
<!--begin.rcode random_param, results='hide', echo=FALSE, message=FALSE
mods <- getModelInfo()
isSeq <- unlist(lapply(mods, function(x) !is.null(x$loop)))
isSeq <- names(isSeq)[isSeq]

mod_list <- paste(sort(paste('<code>', isSeq, '</code>', sep = "")), collapse = ', ')

count_param <- function(x) {
  x <- x$parameter
  if(nrow(x) == 1 && all(x$parameter == "parameter")) return(0)
  nrow(x)
}
nparam <- unlist(lapply(mods, count_param))
    end.rcode-->
     
      
<p>
The default method for optimizing tuning parameters in <span class="mx funCall">train</span> is to use a <a href="training.html#grids">grid search</a>. This approach is usually effective but, in cases when there are many tuning parameters, it can be inefficient. An alternative is to use a combination of <a href="adaptive.html">grid search and racing</a>. Another is to use a <a href="http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf">random selection of tuning parameter combinations</a> to cover the parameter space to a lesser extent. 
</p>
      
<p>
There are a number of models where this can be beneficial in finding reasonable values of the tuning parameters in a relatively short time. However, there are some models where the efficiency in a small search field can cancel out other optimizations. For example, a number of models in caret utilize the "sub-model trick" where <i>M</i> tuning parameter combinations are evaluated, potentially far fewer than M model fits are required. This approach is best leveraged when a simple grid search is used. For this reason, it may be inefficient to use random search for the following model codes: <!--rinline I(mod_list) -->
</p>      
 
<p>
Finally, many of the models wrapped by <span class="mx funCall">train</span> have a small number of parameters. The average number of parameters is <!--rinline round(mean(nparam), 1) -->. 
</p>

<p>
To use random search, another option is available in <span class="mx funCall">trainControl</span> called <span class="mx arg">search</span>. Possible values of this argument are <tt><span class="hl str">&quot;grid&quot;</span></tt> and <tt><span class="hl str">&quot;random&quot;</span></tt>. The built-in models contained in caret contain code to generate random tuning parameter combinations. The total number of unique combinations is specified by the <span class="mx arg">tuneLength</span> option to <span class="mx funCall">train</span>.  
</p>

<p>
Again, we will use the sonar data from the previous training page to demonstrate the method with a self-organizing map by looking at a total of 30 tuning parameter combinations:
</p>

    
<!--begin.rcode random_nnet, tidy=FALSE,cache=TRUE
library(mlbench)
data(Sonar)

library(caret)
set.seed(998)
inTraining <- createDataPartition(Sonar$Class, p = .75, list = FALSE)
training <- Sonar[ inTraining,]
testing  <- Sonar[-inTraining,]

fitControl <- trainControl(method = "repeatedcv",
                           number = 10,
                           repeats = 10,
                           classProbs = TRUE,
                           summaryFunction = twoClassSummary,
                           search = "random")

set.seed(825)
som_fit <- train(Class ~ ., data = training, 
                  method = "xyf", 
                  preProc = c("center", "scale"),
                  metric = "ROC",
                  tuneLength = 30,
                  trControl = fitControl)
som_fit
    end.rcode--> 
    
<p>
There is currently only a <span class="mx funCall">ggplot</span> method (instead of a basic <span class="mx funCall">plot</span> method). The results of this function with random searching depends on the number and type of tuning parameters. In this case, it produces a scatter plot of the three continuous parameters and colors the points based on the topology. The lines are added in the code below. 
</p>
   
<!--begin.rcode random_plot,fig.width=8,fig.height=6
ggplot(som_fit) + 
  geom_smooth(se = FALSE, span = .8, method = loess) + 
  theme(legend.position = "top")
    end.rcode-->   
   
<p>
Otherwise, the <span class="mx funCall">train</span> object works just as any other object from that function.
</p>
   
<div style="clear: both;">&nbsp;</div>
  </div>
  <!-- end #content -->
<div id="sidebar">
<ul>
  <li>
    <h2>General Topics</h2>
    <ul>
      <li><a href="index.html">Front Page</a></li>
      <li><a href="visualizations.html">Visualizations</a></li>
      <li><a href="preprocess.html">Pre-Processing</a><li>
      <li><a href="splitting.html">Data Splitting</a></li>
      <li><a href="varimp.html">Variable Importance</a></li>
      <li><a href="other.html">Model Performance</a></li>
      <li><a href="parallel.html">Parallel Processing</a></li>
    </ul>
    <h2>Model Training and Tuning</h2>
    <ul>
      <li><a href="training.html">Basic Syntax</a></li>
      <li><a href="modelList.html">Sortable Model List</a></li>
      <li><a href="bytag.html">Models By Tag</a></li>
      <li><a href="similarity.html">Models By Similarity</a></li>
      <li><a href="custom_models.html">Using Custom Models</a></li>
      <li><a href="sampling.html">Sampling for Class Imbalances</a></li> 
      <li><a href="random.html">Random Search</a></li> 
      <li><a href="adaptive.html">Adaptive Resampling</a></li> 
    </ul>
    <h2>Feature Selection</h2>
    <ul>
      <li><a href="featureselection.html">Overview</a>
      <li><a href="rfe.html">RFE</a></li>
      <li><a href="filters.html">Filters</a></li>
      <li><a href="GA.html">GA</a></li>
      <li><a href="SA.html">SA</a></li>
    </ul>  
  </li>
</ul>
</div>
<!-- end #sidebar -->
<div style="clear: both;">&nbsp;</div>
  </div>
  <div class="container"><img src="images/img03.png" width="1000" height="40" alt="" /></div>
  <!-- end #page -->
</div>
  <div id="footer-content"></div>
  <div id="footer">
  <p>Created on <!--rinline I(session) -->.</p>
  </div>
  <!-- end #footer -->
</body>
  </html>
